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ABSTRACT 
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anything to do with presenting valuable data or with educational 
policy, a study was undertaken of the report cards of Southeastern 
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and Tennessee) provided report cards for review. These were analyzed 
for similarities and differences with regard to (l) instruments used 
to measure student performance; (2) student outcomes reported and 
reporting procedures; (3) levels of outcome data reported; (A) school 
and community factors reported; and (5) statistical procedures used 
in evaluating reported data. Evaluation reveals that analysis and 
presentation of outcome data are not consistent from state to state 
and appear to reflect the dictates of state policy. Student, school, 
and community characteristics reported also vary, and little attempt 
is made to determine relationships between characteristics and 
achievement. Only Florida and North Carolina attempt to use factors 
other than test results as indicators of student performance. Three 
tables and two figures present comparat ive results . (SLD) 
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AN ANALYSIS OF STATE REPORT CARDS ON SCHOOLS 
PRODUCED IN FIVE SOUTHEASTERN STATES 

by Russell L. French and Gordon C. Bobbett* 

I . INTRODUCTION 

"Report cards" on schools have become common in many states. Their 
contents and formats vary from state to state. A cursory examination of 
the different reports suggests that the variations may have little to do 
with presenting data that are valuable to educators, policymakers, and 
parents in improving education and much to do with policy initiatives and 
the politics of education within the state. However, little detailed 
examination and comparison of report cards have been conducted. There 
is reason to believe that such an investigation could be useful to a 
number of persons. That assumption led to the study reported here. 

II. METHODOLOGY 

Requests for copies of report cards and explanatory information were 
made to all Southeastern states known to be publishing or developing 
report cards. Five states provided materials that were useable- They 
were Florida, Georgia, Mississippi, North Carolina and Tennessee. 

Each report card and the accompanying information were analyzed for 
similarities and differences in five categories: 1) instruments used to 
measure student performance, 2) student outcomes reported and the 
procedures for reporting them, 3) levels of outcome data reported; i.e., 
district, school, grade level, classroom, 4) school and community factors 
reported, and 5) statistical procedures used in evaluating the data 
reported. Findings of the study are reported in each of these five 
categories . 



*Gordon Bobbett is an educational consultant currently living in 
Knoxville, Tennessee. Russell French is professor of Curriculum and 
Instruction, The University of Tennessee, Knoxville. 



III. FINDINGS 
Instruments Used To Measure Student Performance 

As might be expected, instruments and procedures used to measure 
student performance differ from state to state. Table 1 displays the 
findings : 



Table 1. Instruments Used to Measure Student Performance 



STATE 


INSTRUMENTS 


COMMENTS 


Florida 


Grade Ten Assessment Test 
(GTAT) (Reading 
Comprehension, Math) 


Percentage of students 
below the 25th percentile 
and above the 75th 
percentile reported . 




American College. Test 
(ACT) 


Percentage of students (by 
gender and race) taking 
test and median score for 
school reported . 




Scholastic Aptitude Test 
(SAT) 


Percentage of students (by 
gender and race) taking 
test and median score for 
school reported . 




Average Number of Students 
Per Computer 


Used as an indicator of 
readiness to use 
technology. 




Completion of Upper Level 
Science and Math Courses 


Percentage of students (by 
gender and race) reported 


Georgia 


Curriculum Based 
Assessment (CBA) , Grades 
3, 5, 8 (Language Arts, 
Reading, Math, Science, 
Social Studies, Health) 


Matrix sampling procedure 
used; scores reported by 
percentage of students in 
each quartile. 




Iowa Test of Basic Skills, 
Grades 3, 5, 8 


Percentage of students in 
each quartile reported. 




Tests of Achievement and 
Proficiency (TAP) Grade 11 
(Reading, Math, Written 
Expression, Science, 
Social Studies) 


Reported in grade 
equivalents . 

NOTE: All scores are 
reported in 19 school 
system groupings based on 
school district size and 
percentage of students on 
free/reduced lunch. 


Mississippi 


Basic Skills Assessment 
Program (BSAP) , Grade 5 
(Math, Reading, Written 
Communi c a t i on , Compos i t e ) 


Reported as mean scaled 
scores for district and 
school 
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North 
Carolina 



Tennessee 



Functional Literacy Exam 
(FLE) , Grade 11 (Reading, 
Math, Written 
Communication, Composite) 

Subject Area Testing 
Program (SATP) , Algebra I 

Stanford Achievement Test 
(SAT) , Grades 4, 6, 8 



California Achievement 
Test (CAT), Grades 3, 6, 8 
(Reading/Language , Math) 

N. Carolina Tests, Grades 
3 , 6, 8 (Writing, Social 
Studies , Science) 

NOTE : Writing test 
administered only at 
grades 6 and 8 

North Carolina Tests, High 
School (Economics/ 
Legal/Politics , Biology, 
Chemi s t ry , Phy s i c s , 
Physical Science , Algebra 
I, Algebra II, Geometry) 

Scholastic Aptitude Test 
(SAT) 

Advanced Placement 
Examinations 

Percentage of students in 
Grades 9-1 ~ Earning 5 or 
more units toward 
graduation 

Percentage of Graduates 
completing required UNC 
Admi s s i ons Cou r se s 

Tennessee Comprehensive 
Assessment Program (TCAP) , 
Grades 2 thru 8 and 10 
(Reading, Language , Math, 
Science , Social Studies) 



Tennessee Proficiency Test 
(TPT) , Grade 9 



Same procedure as BSAP 



Same procedure as BSAP and 
FLE 

Reported in terms of mean 
national normal curve 
equivalent for system and 
school . 

Reported by percentage of 
students at each 
percentile in the 
district . 

Reported for current year 
and past two years in 
percentiles 



Same reporting procedure 
as Grade 3, 6, 8 tests 



Average scores by district 



Number of students scoring 
3 or above 



Formerly reported as 
average percentile at each 
grade level; now reported 
in terms of average gain 
over two years and 
percentage of gain (plus 
or minus) against national 
norm . 

Reported as percentage of 
students passing test 
(required score of 70 
percent) 
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Analysis of this table indicates that all five states use state- 
developed tests to measure aspects of student academic performance. All 
of the states except Tennessee report scores from at least one recognized 
national achievement test; e.g., Stanford Achievement Test, Iowa Test of 
Basic Skills, California Achievement Test. Test scores/ results are 
presented differently in each state, and in two states (Florida, North 
Carolina) , indicators other than test scores are included as measures of 
performance . 

Student Outcomes Reported 

Table 1 also provides the information necessary for comparison of 
student outcomes reported in the five states. Florida reports 
per jentages of students scoring below the 25th percentile and above the 
75th percentile on the Grade Ten Assessment Test. Median scores on the 
ACT and SAT examinations are used. Georgia reports percentage of 
students by quartile on its Curriculum Based Assessment and the Iowa Test 
of Basic Skills, but reports subject area test scores (TAP) in grade 
equivalents. Mississippi uses a mean scaled score reporting format for 
its BSAP, FLE and SATP testing programs, but the state reports Stanford 
Achievement Test scores in terms of what it calls a mean national normal 
curve equivalent. North Carolina reports percentages of students at each 
percentile level except for the Scholastic Aptitude Test (average 
district scores) and Advanced Placement Examination results (number of 
students scoring 3 or above) . Tennessee reported average percentile 
score at each grade level for each test in its comprehensive Assessment 
program (TAP) until 1992-93 and the percentage of students passing the 
Tennessee Proficiency Test (scores of 70 percent or above) . The 
reporting procedure has changed with the advent of the Tennessee Value 
Added Assessment Program (TVAAS) . 

Both Tennessee and North Carolina use test results to make summative 
evaluation decisions about school districts. However, each state 
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approaches its evaluation differently. The North Carolina report card 
provides four comparisons of student performance for each school system: 

• comparison of current levels of student performance with those 
of previous years, 

• comparison of performance of the school system with all other 
school systems in the state, 

• comparison of performance of the school system with similar 
school systems in the state, 

• comparison of current levels of student performance with state 
accreditation standards. 

Some measures also allow comparison of the performance of a school 

system's students with that of students nationwide. In North Carolina's 

approach, school system and community characteristics are used to 

calculate an index of advantagement . This index, which takes the form 

of a positive or negative number, is the vehicle for comparison of 

educational outcomes in similar school districts. The comparison of 

student performance with state accreditation standards is accomplished 

by summarizing school system test scores into four curriculum areas 

(mathematics, reading/language, science, social studies), deriving a 

single system achievement score for each curriculum area and, ultimately, 

a single overall achievement score for the school system, a score 

representing achievement across all curriculum areas. It is then 

possible to determine whether student performance in a school system is 

average, below average or above average and to determine the system's 

level of achievement of each of 34 state performance standards which are 

the basis for school accreditation. Figure 1 provides examples. 

When fully implemented, Tennessee's value added assessment approach 

will result in rewards and penalties to school and school systems based 

on performance gains over a minimum of two years. The procedure used to 

compute gains is too complex to be fully explained here. In essence, 

estimated mean gain of a group of students in a specific subject is 

produced from mixed model equations. That gain is then compared with 
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Figure 1. North Carolina Presentation 
of Achievement Data 

Achievement by Subject Area 
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Figure 1 (continued) 
'Ntfrth Carolina Presentation 
of Achievement Data 

Standard 



BUNCOMBE COUNTY 
1992 ACCREDITATION STATUS 



110 



1. (1.1) Attendance 



6. (2.5) 

7. (3.1) 

8. (3.2) 

9. (3.3) 



10. 

11. 

12. 

13, 

14. 

15. 

16. 

17. 

18. 

19. 
20. 
21. 
XL 
23, 
24. 
25. 
26. 

27. 
28. 
29, 
30. 
31. 
32. 
33. 
34. 



(2.1) 5 Unit* of Credit for Graduation 

(2.2) Entry to UNC Institutions 

(2.3) Qualify for NC Scholars Program 

(2.4) Voc. Ed. Unemployment Rale 

County Youth Unemployment 
Follow-Up Survey Response Rate 

Percent Certificates 
Compensatory Reading 
Compensatory Math 
Dropouts 

CAT, 3rd Grade Reading 
CAT. 3rd Grade Language 
CAT, 3rd Grade Mathematics 
CAT, 6th Grade Reading 
CAT, 6th Grade Language 
CAT, 6th Grade Mathematics 
CAT, 8th Grade Reading 
CAT, 8th Grade Language 
CAT, 8th Grade Mathematics 
Writing Essay, 6th Grade 
Writing Essay, 8th Grade 
Science, 3rd Grade 
Science, 6th Grade 
Science, 8th Grade 
Social Studies, 3rd Grade 
Social Studies, 6th Grade 
Social Studies, 8th Grade 

Algebra I 
Algebra II 
Biology 

Uniteu States History 
Chemistry 
Geometry 
English 
Physics 

SUMMARY 
Standards Fully Met (•♦♦•) 
Standards Met Level 1 (•♦♦) 
Standards in Warning Status (••) 
Standards Not Met (•) 



(4.1a) 
(4.1b) 
(4.1c) 
(4.2a) 
(4.2b) 
(4.2c) 
(4.3a) 
(4.3b) 
(4.3c) 
(4.4) 
(4.5) 
(4.6) 
(4.7) 
(4.8) 
(4.9) 
(4.10) 
(4.11) 

(5.1) 
(5.2) 
(5.3) 
(5.4) 
(5.5) 
(5.6) 
(5.7) 
(5.8) 



Criterion 


Performance 


Level of 
Compliance, 


C\ a to 


93.92 


• 


80% 


88.8 


• 


35% 


51 


•••• 


10% 


20.5 


•••• 


£ County 


5.2 


•••• 




16.4 






oy.o 




£3% 


1.4 




1NCE 


8.4 




1 NCE 






£ 2.4% 


3.63 


•••• 


40-50%iIe 


JJ.7 




40-50%ile 


60 5 


•••• 


40-50%iIe 


76.3 




40-50%ilc 


57 




40-50%ile ■ 


59.5 




40-50%iIe 


69.5 




40-50%ilc 






40-50%iIe 


61 0 




40-50%ile 






40% 


4fi 7 




40% 


A1 7 




40-50%ilc 


66 




40-50%ile 


60.2 




40-50%ile 


62.8 


•••• 


40-50%ile 


65.6 




40-50%iJe 


55 




40-50%ile 


58.4 




40-50%ile 


WtO 




40-50%iIe 




•••• 


40-50%ile 


65.4 




40-50%ile 


56.4 




40-50%iIe 


60 




40-50%Uc 


63.5 




40-50%iJe 


54.2 




40-50%ile 


48,5 





NUMBER 
31 
1 
0 
1 



PERCENT 
93.9 
3 
0 
3 
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national norm gains, and the relationship of local gains to national 
gains is determined using scale score points. Bar graphs vividly present: 
to the school or school system its comparative gain at each grade level 
in relation to national norms. Figure 2 provides an illustration. 
Levels of Outcome Data Reported 

The five state report cards differ in the levels of information 
presented as indicated in Table 2. 



Table 2. Levels of Data Presented In State Report Cards 



State 


Performance Data 


School/District Characteristics 


Florida 


District Level 
School Level 
Grade Level* 


District Level 
School Level 


Georgia 


District Level 
Grade Level* 


District Level 


Mississippi 


District Level 
School Level 
Grade Level* 


District Level 


North Carolina 


District Level 
School Level 


District Level 


Tennessee 

i 


District Level 
School Level 
Grade Level 


District Level 



*Grade level data provided for tests given only at specified levels. 



Only one state (Georgia) does not provide school level performance data, 
and only one state (Tennessee) provides grade level performance data for 
all grade levels two through ten. Only Florida provides information in 
its report cards about student and school characteristics at the school 
as well as the district level. 
School and Community Characteristics 

School and community characteristics presented in the five state 
report cards also differ. However the characteristics presented can be 
clustered in categories: student characteristics, school/district 
characteristics, financial characteristics of the community. Table 3 
provides the comparison: 
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Figure 2. Tennessee, Value Added Assessment Presentation 

ILLUSTRATION Math 
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• Indicates 8th Grade Math 
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Table 3: Student, School And Community Characteristics 
Identified In Report Cards 



State 



Florida 



Student 

Characteristics 



•Racial distribution 
(White, Black, 
Hispanic , Asian, 
Indian) 

•Percent 

free/reduced lunch 

•Percent gifted 

•Percent handicapped 

•Percent in federal 

compensatory 

programs 

•Percent limited 
English Proficient 
(by race) 

•Percent habitual 
truants 



School/District 
Characteristics 



•Percent 

kindergarten 
retention 

•Percent first grade 
retention 

•Graduation rate 

•Student mobility 
(%) 

•Student attendance 
(%) 

•Percent students 
promoted/ K-3 

•Percent students 
promoted, 4-6 

•Percent in school 
suspensions 

•Percent out of 

school suspensions 

•Percent corporal 
punishment 

•No full time 
teachers and staff 

•Racial /Ethnic 
composition of staff 

•Percent teachers by 
degree levels 

•Percent teachers by 
experience levels 

•Staffing ratios 
(pupils per teacher, 
pupils per 
administrator, 
pupils by librarian) 

•Instructional staff 
per administrator 



Community/District 

Financial 

Characteristics 



•Per pupil 
expenditure 

•District funding 
by source (local, 
state, federal) 
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Georgia 
Mississippi 



North Carolina 



•Percent 

fr j/reduced lunch 

•^ ;rcent race 
(black, white) 

•Percent gender 

•Percent limited 
English proficient 

•Percent handicapped 

•Number and percent 
race (American 
Indian, Asian, 
Hispanic , Black, 
White) 

•Percent gifted 
•Percent handicapped 

•Percent in 

compensatory 
education programs 

•Percent 

free/reduced lunch 

•Percent absent more 
than 14 days 



•School system size 



•Average daily 
attendance 



•Membership (number 
of students) 

•Average number of 
students per teacher 

•Percent teachers 
with graduate 
degrees 

•Number of high 
school completers 

•Number of 

vocational education 
completers 

•Number of NC 

scholars program 
course completers 

•Number of students 
taking AP exams 

•Number of students 
in grades 9-12 
earning 5 or more 
units toward 
graduation 

•Number of graduates 
completing UNC 
required Admissions 
Courses 



•Local per pupil 
expenditures 

•Total per pupil 
expenditures 

•Average local 
teacher salary 
supplement 

•Parent education 
level (percent 8th 
grade, 8-12, high 
school graduates, 
post high school) 
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Tennessee 



•Percent 

free/reduced lunch 

•Percent in special 
education 

•Percent chapter I 
students 



•Number of schools 

•Average dai ly 

membership 

•Percent student 
attendance 

•Percent enrollment 
change 

•Percent oversized 
classes 

•Percent elementary 
schools accredited 
by SACS 

•Percent educators 
on Career Ladder 
Levels II and III 

•Average 

professional 
educator salary 

•Percent diplomas 
granted ( regular , 
honors , special 
education, 
certificate of 
attendance) 

•Percent students in 
vocational education 
courses 



•Average 

expenditure per 
pupil 

•County per capita 
income 



Analysis of Table 3 indicates that all five states report some range 
of student characteristics, and three states (Florida, North Carolina, 
Tennessee) report community/district financial characteristics. Georgia 
and Mississippi focus their reports on academic outcomes, presenting only 
a limited amount of additional information which they feel is needed to 
present and interpret test scores . 
Statistical Procedures Used In Evaluating Data 

As already reported in Table 1, the five state report cards analyzed 
use a variety of statistical procedures in reporting student outcomes. 
Florida reports percentile scores for its Gjrade Ten Assessment Test and 
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median school score for the ACT and SAT. Georgia uses quartiles for its 
Curriculum Based Assessment and the Iowa Test of Basic Skills., but 
computes grade equivalents for its Tests of Achievement and Proficiency. 
Mississippi computes mean scaled scores for all tests except the Stanford 
Achievement Test for which national normal curve equivalents are 
computed. North Carolina determines percentile rankings for all test 
results except the SAT and Advanced Placement examinations which do not 
lend themselves to this analysis . Tennessee formerly reported mean 
percentile scores for each grade level , but now uses deviation from 
national norm gain. 

None of the report cards studied report statistical analyses of the 
impact of student, school or community factors on student outcomes. In 
fact, there is no indication that statistical analyses of these 
relationships are being conducted. North Carolina states in its 
explanatory materials that the advantagement index provided in the report 
card is calculated using factors that research has shown to be related 
to srhocl achievement. However, no identification of those specific 
factors or procedure for using them is provided. 

I T \ CONCLUSIONS 

Although the sample of report cards analyzed in this study is small, 
several generalizations can be made: 

1 . There is little commonality from state to state in the 
performance measures and indicators incorporated into current 
state report cards . 

2 . Procedures used in analyzing and presenting student outcome 

data are not consistent from state to state. They appear to 
represent the dictates of state policy or the particular bent 
of report card developers . 

3 . Student, school and community characteristics reported or used 

in interpreting data also vary from state to state. The three 
most commonly used factors are percentage of students on 
free /reduced lunch, student attendance (and its corollary 
student absence), and per pupil expenditures. 

4 . There is little attempt to determine relationships between 
student /school /community characteristics and student 
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performance. There appears to be a tacit assumption that the 
characteristics reported influence outcomes. 

5. Early versions of scate report cards tended to focus at the 
district/system level. These reports draw attention school 
and grade level data. 

6. While several of the reports provide for comparison '.\?ith like 
schools and/or districts, there is no information provided 
that would offer educators and community leaders insights into 
the factors in similar schools that might be contributing to 
higher performance levels, where those exist. For example, 
there is no information about curriculum structure , 
instructional methodologies , educator professional 
development, or school organization and governance. 

7. Only a few states (Florida and North Carolina in this study) 
are attempting to use factors other than test results as 
indicators of student performance. 

V. IMPLICATIONS 

Several implications emerge from the findings and conclusions of 
this study. We offer them as points for discussion. 

The Ne w Standards And Assessments Debate . When viewed in the 
context of the current effort to develop new standards and assessments 
that extend beyond state boundaries, this study suggests that much 
groundwork will need to be done before policymakers and educators are 
willing to "buy in" to r'^ional or national frameworks and procedures. 
These report cards demonstrate clearly that each of several neighboring 
states has approached the task of assessing and portraying schooling and 
student performance independently and differently. It is uncertain that 
they will be willing to compromise their perspectives and practices in 
order to provide a "common view" of schools and student outcomes. 

The Measurement of Student Performance . Without exception, each of 
the five states sampled in this study is using one or more state 
developed tests/test batteries in its assessment package. While there 
are good reasons for the development of these measures, one cannot help 
but acknowledge the time and costs expended by each state. Further, one 
wonders to what extent these state produced measures produce information 
that is any better than assessment^ produced for use nationally. The 
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argument has been that state produced tests more validly reflect the 
curricula within that state. Yet, teachers and administrators in many 
of the states sampled still question the alignment of the tests with 
their curricula. In light of current reform efforts, student and family 
mobility across state boundaries, and the tremendous costs involved in 
development of new assessments, at least one question must be posed, "Is 
it time for interstate collaboration? Can we really afford to reinvent 
the wheel in every state?" 

Report Card Development . As demonstrated in this study, state 
report cards on schorls typically portray school districts and schools 
through a variety of performance indicators and student, school, 
community characteristics. The tacit assumption is that the 

characteristics somehow impact performance. However, there is no 
indication of the actual influence of any characte stic or set of 
characteristics on the outcomes presented. 

In a series of other papers, the authors have demonstrated that few 
of the characteristics usually presented have much impact on student 
academic achievement. In those studies, the factors that most commonly 
influence student performance are attendance and per pupil expenditure. 
However, even these factors influence performance differently at 
different grade levels. 

Further, as indicated in the conclusions of the present study, there 
has been little attempt to build into report cards information that might 
be useful to educators desiring to improve performance in their schools. 
As a teacher or school administrator, I can compare the performance of 
my school with others, but I have no idea why their performance might be 
better. 

In essence, it may be time to relook at the structure and content 
of school report cards. They can be an extremely useful tool for 
improvement, but that potential is not being reached. 
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